On the surplus value of semantic video analysis beyond the key frame

Typical semantic video analysis methods aim for classification of camera shots based on extracted features from a single key frame only. In this paper, we sketch a video analysis scenario and evaluate the benefit of analysis beyond the key frame for semantic concept detection performance. We developed detectors for a lexicon of 26 concepts, and evaluated their performance on 120 hours of video data. Results show that, on average, detection performance can increase with almost 40 % when the analysis method takes more visual content into account. 1

UvA-DARE (Digital Academic Repository) A Physical Basis for Color Constancy

A Physical Basis for Color Constancy Geusebroek, J.M.; van den Boomgaard, R.; Smeulders, A.W.M.; Gevers, T. General rights It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons). Disclaimer/Complaints regulations If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible. Abstract A fundamental problem in psychophysical experiments is that significant conclusions are hard to draw due to the complex experimental environment necessary to examine color constancy. An alternative approach to reveal the mechanisms involved in color constancy is by modeling the physical process of spectral image formation. In this paper, we aim at a physical basis for color constancy rather than a psychophysical one. By considering spatial and spectral derivatives of the Lambertian image formation model, object reflectance properties are derived independent of the spectral energy distribution of the illuminant. Gaussian spectral and spatial probes are used to estimate the proposed differential invariant. Knowledge about the spectral power distribution of the illuminant is not required for the proposed invariant. The physical approach to color constancy offered in the paper confirms relational color constancy as a first step in color constant vision systems. Hence, low-level mechanisms as color constant edge detection reported here may play an important role in front-end vision. The research presented raises the question whether the illuminant is estimated at all in pre-attentive vision

An Image Statistics–Based Model for Fixation Prediction

The problem of predicting where people look at, or equivalently salient region detection, has been related to the statistics of several types of low-level image features. Among these features, contrast and edge information seem to have the highest correlation with the fixation locations. The contrast distribution of natural images can be adequately characterized using a two-parameter Weibull distribution. This distribution catches the structure of local contrast and edge frequency in a highly meaningful way. We exploit these observations and investigate whether the parameters of the Weibull distribution constitute a simple model for predicting where people fixate when viewing natural images. Using a set of images with associated eye movements, we assess the joint distribution of the Weibull parameters at fixated and non-fixated regions. Then, we build a simple classifier based on the log-likelihood ratio between these two joint distributions. Our results show that as few as two values per image region are already enough to achieve a performance comparable with the state-of-the-art in bottom-up saliency prediction

SIGNIFICANCE OF THE WEIBULL DISTRIBUTION AND ITS SUB-MODELS IN NATURAL IMAGE STATISTICS

Natural image statistics, Weibull distribution, model selection. The contrast statistics of natural images can be adequately characterized by a two-parameter Weibull distribution. Here we show how distinct regimes of this Weibull distribution lead to various classes of visual content. These regimes can be determined using model selection techniques from information theory. We experimentally explore the occurrence of the content classes, as related to the global statistics, local statistics, and to human attended regions. As such, we explicitly link local image statistics and visual content.